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The Relationship Between MSPAP and Social Studies 
Classroom Instruction and Assessment Materials 



As part of an educational reform movement that began in the 1980’s, a task force was 
formed to develop curriculum standards for the social studies subject area (National Council for 
the Social Studies Bulletin, 1994). The standards were developed to ensure that students would 
be taught relevant and appropriate content at certain points in their education. The standards 
provide a guide for instruction and assessment practices that can be implemented in individual 
states, districts, and schools to promote a solid, integrated, and active education for students 
(National Council for the Social Studies Bulletin, 1994). 

Many states have set similar standards as the basis for developing specific learning goals 
for their own students. One such state is Maryland, which began its implementation of a 
statewide performance assessment in the early 1990’s. The Maryland State Performance 
Assessment Program (MSPAP) is based on a performance assessment designed to measure school 
performance for grades 3, 5, and 8 in the state of Maryland (Maryland State Board of Education, 
1995). The MSPAP was developed based on a set of learning outcomes developed by educators 
in that state. 

The Maryland State Department of Education, in conjunction with Maryland social 
studies educators, determined the learning outcomes and indicators of these outcomes that 
MSPAP would assess for the social studies subject area. These learning outcomes deal with the 
understanding of content and abilities related to social studies and are assessed by MSPAP. 
MSPAP is intended to promote performance-based instruction and assessment practices based on 
the Maryland Learning Outcomes. 

As part of a research project being conducted to determine the impact that the Maryland 
Learning Outcomes and MSPAP are having on classroom instruction and assessment practices, 
social studies classroom materials were collected from elementary and middle schools in the state 
of Maryland during the 1997-98 school year. These classroom materials were examples of 
instruction and assessment activities used by teachers in their day-to-day teaching. The activities 
were analyzed with respect to the extent to which they are aligned with the Maryland Learning 
Outcomes in the social studies subject area, and the extent to which they represent MSPAP tasks 
with regard to such features as processes assessed, format, response types required, and 
integration with other subject areas. This paper presents the findings of the analyses. 
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Methodology 



Sample 

A stratified random sampling procedure was used to select schools that would participate in 
the study, with the strata being defined by three levels of the following: a) percent free or 
reduced lunch according to the 1994-95 classification, and b) MSPAP performance gains 
(MSDE’s 1993-95 change index). Schools were classified into one of the nine cells based on 
their rankings in the distributions for these two variables. A random subset of schools in the 
entire sample were asked to send in classroom materials. Overall, 74 schools with 428 teachers 
were asked to participate in this aspect of the study. Some or all of the teachers from 61 of the 
schools participated, resulting in a school participation rate for classroom activities of 82%. This 
represents schools from 15 different systems/counties in Maryland. Within these 61 schools, 304 
social studies teachers sent in all or a subset of the activities requested (71%). 

Data Collection and Instruments 

After being selected for participation in this study, social studies teachers were asked to send 
in approximately 3 to 4 instruction activities, 3 to 4 assessment activities, and 1 sample of a 
scoring scheme used in their classrooms from September to December 1997. Similarly, they 
were asked to send in another set of 3 to 4 instruction activities, 3 to 4 assessment activities, and 1 
sample of a scoring scheme used from January to June 1998. These activities would then 
represent classroom activities used by the teachers in both the fall and spring of their school year. 
In addition, 3^“*, 5*, and 8* grade social studies teachers were asked to send a sample of a MSPAP 
test preparation activity used in the spring, prior to the administration of MSPAP. If a teacher 
taught more than one social studies class, the teacher was requested to obtain these materials from 
a typical class. A total of 304 social studies teachers sent in a sample of social studies classroom 
activities used during the 1997-98 school year. 

A data collection form was developed to obtain information from each teacher regarding the 
classroom from which the activities were selected. The data collection form asked teachers to 
indicate the grade level, the nature of the students’ ability levels for the social studies class (e.g., 
heterogeneous ability group, homogeneous ability group, exclusively special education, 
exclusively gifted and talented), and the nature of the content taught in the class (e.g., history, 
geography, world cultures, economics). Each teacher completed a form and returned it along 
with the activities for each collection period. 

Teachers were also provided with a set of labels to attach to their classroom activities. On 
these labels teachers were instructed to indicate whether the activity was used for instruction 




2 



purposes, assessment purposes, or scoring/evaluating purposes. They also were asked to indicate 
the source of the activity (e.g., teacher developed, commercial resource or textbook, county- 
developed, state-developed). In this paper, only those activities that were used for instruction and 
assessment purposes are discussed. 

Procedures for Coding Social Studies Classroom Activities 

The classroom instruction, assessment, and MSPAP test preparation activities were analyzed 
using a coding scheme designed to provide information about the extent to which the activities 
reflected the Maryland Learning Outcomes for social studies, their overall similarity to MSPAP- 
like tasks, and a variety of other features (e.g., response type required of students, integration with 
other subject areas, etc.). The Maryland Learning Outcomes and the format and content of 
MSPAP served as the basis for the coding schemes that were developed for the analysis of the 
classroom activities. 

A total of five raters coded the classroom activities. A formal training session was conducted 
to familiarize the raters with the coding scheme using a set of pre-coded activities. Then, the 
raters coded another set of activities independently and their codes were compared and discussed 
by the group. After the formal training was complete, pairs of raters individually coded sets of 
classroom activities from a school (elementary or middle) for a certain collection period (fall or 
spring). The pair of raters met to discuss their discrepancies and reached a consensus on the 
codes for each activity. This was done to ensure that all raters shared a common understanding of 
the coding scheme. Thus, for a small percentage of classroom activities, one set of codes, agreed 
upon by two raters, was obtained. 

Description of Social Classes and Teachers 

A total of 304 social studies teachers sent in a sample of their classroom activities. The 
following sections provide a description of the type of social studies classroom from which 
activities were obtained, the student heterogeneity of social studies classes, information on 
sample sizes by grade and type of activity, and the sources of the activities. 

Type and Student Heterogeneity of Social Studies Class 

Teachers were asked to indicate the type of social studies class from which their sample of 
classroom activities was selected. A large majority of the classes (70%) were ‘general social 
studies’ classes. Smaller percentages of the classes were history (6%), geography (2%), or world 
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cultures (5%). The type of class from which the remaining 17% of the classroom materials were 
obtained was not indicated. 

Teachers were also asked to indicate the heterogeneity of the ability level of the students in 
the social studies classes from which their samples of classroom activities were selected. The 
majority of classes were heterogeneous (62%), and only 12% of the classes were homogeneous. 
Even smaller percentages of the classes were ‘exclusively special education’ (8%) or ‘exclusively 
gifted and talented’ (3%). The student heterogeneity of 14% of the classes was not indicated. 

Sample Sizes for Teachers and Classroom Activities by Grade Level 

On average across the entire school year, approximately 12 classroom instruction and 
assessment activities were collected per teacher. In the fall, 278 social studies teachers sent in 7 
classroom activities on average, and in the spring, 192 social studies teachers sent in an average 
of 8 classroom activities. The reason for the discrepancy between the overall average number of 
classroom activities received and the average number of activities for fall and spring considered 
separately is that 138 teachers sent in classroom activities for either the fall or spring, but not 
both. For each grade level. Table 1 indicates the number and percentage of teachers who sent in 
classroom activities and also presents the total number and percentage of all classroom activities 
received. For example, 42 2"“* grade teachers sent in a total of 471 classroom activities. The 
percentages across grades for the number of teachers and the number of activities are similar. 



Table 1. 

Number of Teachers and Classroom Activities by Grade Level 



Grade 


Teachers 


Activities 




Number 

(n=305*) 


Percentage 


Number 

(n=3542) 


Percentage 


2 


42 


14% 


471 


13% 


3 


59 


19% 


686 


19% 


4 


52 


17% 


525 


15% 


5 


56 


18% 


652 


18% 


7 


44 


14% 


569 


16% 


8 


52 


17% 


639 


18% 



* Note: This number is larger than the 304 teachers who sent in classroom activities because 1 teacher had a change in 
grade taught from fall to spring. 



Sample Sizes by Type of Classroom Activity 

Teachers were provided with labels to attach to each activity that would indicate the type of 
activity (e.g., instruction, assessment, test preparation, scoring scheme). Table 2 shows the 
number and percentage of activities for each type. Across all grades (2, 3, 4, 5, 7, and 8), there 
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were a total of 1433 instruction activities, 1095 assessment activities, and 709 scoring schemes. 
For grades 3, 5, and 8 there were a total of 103 MSPAP test preparation activities. The table also 
includes a category called ‘not coded’ . Activities placed in this category were not coded for one 
of two reasons. One reason for not coding an activity was because it pertained strictly to another 
content area such as language arts or science. Another reason an activity was not coded was 
because the activity consisted only of teacher notes or general lesson plans, and it could not be 
determined what the students were required to do. 

The last three columns of Table 2 indicate the number and percentage of teachers providing 
each type of material as well as the average number of each type of activity sent in per teacher. 
As an example, 41% (1433 out of 3542) of the activities received were labeled as instruction, and 
nearly all teachers (96%) sent in at least one instruction activity. The mean number of instruction 
activities provided per teacher was approximately 5. It should be noted that the percentages for 
the grade levels for each type of activity were similar. It should also be recalled that although the 
percentages for scoring schemes are presented in this table, the results of the scoring scheme 
analysis will not be presented in the current paper. 

Table 2. 

Type of Classroom Activity 





Activities 

(n=3542) 


Teachers 

(n=304) 


Mean Number of 
Activities Per 
Teacher 




Number 


Percentage 


Number 


Percentage 




Instruction 


1433 


41% 


292 


96% 


4.91 


Assessment 


1095 


31% 


271 


89% 


4.04 


MSPAP Test Preparation 


103 


3% 


58 


35%* 


1.78 


Scoring Scheme 


709 


20% 


240 


79% 


2.95 


Not Coded 


202 


6% 


100 


33% 


2.01 



* Note: This percentage is based only on the 167 on-grade teachers (3, 5, 8) and not of the full sample of 304 teachers. 



Sources of Classroom Activities 

Teachers were also asked to indicate the source from which each activity was obtained. For 
instruction, assessment, and test preparation activities. Table 3 indicates across all grades the 
number and percentage of activities that were from a variety of sources. In general, more of the 
instruction, assessment, and test-preparation activities were teacher-developed than were 
developed by any other source. This holds especially for the assessment activities, the majority 
of which were teacher-developed (52%). The percentage of instruction activities that was 
teacher-developed is similar to the percentage that was obtained from a textbook/commercial 
resource, 37% and 31%, respectively. Similar percentages of each type of activity, ranging from 
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11-14%, were county-developed. The percentages of instruction and assessment activities 
obtained from state-level materials, such as MSPAP Release Tasks, Maryland Consortium Tasks, 
and Maryland Performance-Based Exemplars, were very small. However, the percentage of test- 
preparation activities obtained from these sources was higher (17%). 



Table 3. 

Sources of Classroom Activities 





Instruction 

n=1433 


Assessment 

n=1095 


MSPAP 
Test Prep 
(3^“, 5*, 8*) 
n=103 


Teacher/Other Teacher/ School Developed 


37% 


52% 


38% 


Textbook/Commercial Resources 


31% 


20% 


18% 


County/Another County Developed 


14% 


11% 


13% 


Teacher and Textbook/ Teacher and County Developed 


10% 


8% 


11% 


MSPAP Release Tasks/ MD Consortium/ Exemplars 


1% 


1% 


17% 


Other/ Teacher and Student/Class (Scoring) 


1% 


2% 


1% 


Cannot Be Determined 


7% 


8% 


4% 



Results 



As was stated above, the classroom activities were analyzed using a coding scheme designed 
to provide information about the extent to which the activities reflected the Maryland Learning 
Outcomes for social studies and the similarity of the activities to the MSPAP tasks. The learning 
outcomes to be discussed will be classified as Process Learning Outcomes, which deal with the 
thought processes and understanding that students need in order to complete the activities, and 
Content Learning Outcomes, which relate to the content covered by the activities. Other features 
of the classroom activities, including the format of the activity, the type of response required of 
students, and the integration of the activity with other subject areas will be discussed in order to 
relate these features of the classroom activities to the MSPAP tasks themselves. The MSPAP 
tasks, in general, consist of a set of activities related to the same context or problem situation. 
The activities within the task require that the students display numerous skills related to the 
content areas that the task is assessing. Most often, the tasks for social studies require that the 
student write several paragraphs in response to questions, and within the paragraphs explain and 
justify their answers. Reading, understanding what was read, and elaborating on the reading 
material is an integral part of the MSPAP task. Selected response items are not part of the 
MSPAP tasks. The tasks are set in a realistic context, and require the use of high level thinking 
skills. Further, the tasks are integrated with material learned in other subject areas. Many of the 
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MSPAP tasks also require peer review. Again, the Maryland Learning Outcomes and the format 
and content of MSPAP served as the basis for the development of the coding schemes. The 
results of the coding of the classroom activities and their relationship to the Maryland Learning 
Outcomes and MSPAP tasks will now be presented. 

Classroom Activity Format 

Each of the MSPAP tasks consists of a series of items that are related to the same problem 
context or situation. Therefore, for the classroom instruction, assessment, and test preparation 
activities, if all of the items within an activity were related to the same problem context or 
situation (e.g., a day at the courthouse), the activity was treated as ‘one task with several items 
related to the same context’ . Activities were also treated as one task with several related items if 
they consisted of several short explanations related to the same content area. In this type of task, 
it is clear that the items relate to each other, but they may not necessarily be set in a real-life 
context like most MSPAP items. It should be noted that selected response items were never 
coded in this category. If an activity consisted of several items not related to the same context or 
content area, the activity was treated as ‘distinct non-MSPAP items’ (e.g., a set of selected 
response items, or short answer items not related to the same context). Activities were treated as 
‘only one item’ if they consisted of writing a single paragraph or reading a passage and answering 
a single question. In some cases, an activity could be coded as both consisting of ‘one task with 
several items related to the same context’ and a set of ‘distinct non-MSPAP items’. This would 
be the case if a number of short answer items in a section of the activity were isolated, unrelated 
questions, but the rest of the task was related in context. A task could also be coded as a set of 
‘distinct non-MSPAP items’ and ‘only one item’ if the activity consisted of a set of multiple 
choice or matching items and also required a single but extensive written response. Other 
combinations of activity formats were also possible. Table 4 indicates the percentages of 
instruction, assessment, and test preparation activities that were considered to be a task with 
related items, a set of distinct items, and a sole item. It should be noted that many of the tables in 
this paper, including Table 4, do not include grade level results. This is because the differences in 
the percentages across grades were negligible. 
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Table 4. 

Activity Format for Social Studies Classroom Activities 





Instruction 

n=1433 


Assessment 

n=1095 


MSPAP Test Preparation 
n=103 


One task with several items 
related to same context 


65% 


53% 


91% 


Distinct non-MSPAP items 


30% 


47% 


14% 


Only one item 


18% 


18% 


5% 



Across grades, 65% of the instruction activities consisted of ‘one task with several items 
related to the same context’ and 30% of the activities consisted of a set of ‘distinct non-MSPAP 
items’. A larger percentage of assessment activities (47%), as compared to instruction activities 
(30%), consisted of a set of distinct items. In contrast, only a small percentage of MSPAP test 
preparation activities (14%) consisted of a set of distinct items, while most of the MSPAP test 
preparation activities (91%) consisted of ‘one task with several items related to the same context’. 
As might be expected, the MSPAP test preparation activities tended to be more similar in form to 
the actual MSPAP tasks, as compared to the classroom instruction and assessment activities, with 
91% of these activities being coded as one task with several related items. Across all grades, as 
was stated, a larger percentage of assessment than instruction activities consisted of ‘distinct non- 
MSPAP items’. This could indicate that teachers use MSPAP-like activities more in their 
instruction than in their assessment of students. 

Maryland Process Learning Outcomes 

The classroom instruction, assessment, and MSPAP test preparation activities were coded in 
terms of whether they focused on the Process Learning Outcomes as defined by the Maryland 
Learning Outcomes and MSPAP. The Process Learning Outcomes for social studies are “Skills 
and Processes”, “Valuing Self and Others”, and “Understandings and Attitudes”. Within the 
“Skills and Processes” learning outcome, more specific indicators of this process outcome as 
defined by MSPAP were coded. Contained within the “Skills and Processes” learning outcome 
are such abilities as being able to obtain and use textual information; use print and non-print 
media to obtain information; critically examine issues, events or problems; use decision making 
processes; and interact with others in groups. Considered together, the indicators of the “Skills 
and Processes” learning outcome could be found in most of the classroom activities. For this 
reason, and because the indicators of this learning outcome cover a broad range of skills, each 
indicator of the outcome was coded separately. The “Valuing Self and Others” Process Learning 
Outcome deals with the students’ abilities to attain a positive self-concept, and recognize and 
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appreciate cultures different from their own. Examples of indicators of this process outcome are 
the abilities of students to show acceptance and understanding of others, and to recognize that 
people from other cultures may be different from themselves but should be treated with respect 
and dignity. The “Understandings and Attitudes” Process Learning Outcome relates to the 
students’ understandings and attitudes toward basic human rights. Indicators of this outcome are 
such abilities as analyzing situations illustrating conflicts between conscience and respect for 
authority, distinguishing between the concepts of majority rule and rights of the individuals, and 
analyzing situations in which individuals demonstrate respect and support for the rights and 
dignity of all peoples. Because the indicators of each of these last two Process Learning 
Outcomes are similar to each other, overall categories, rather than each of the indicators, were 
coded for the “Valuing Self and Others” and “Understandings and Attitudes” learning outcomes. 

When coding the process outcomes, two categories were added which do not reflect the 
Process Learning Outcomes as defined by MSPAP. They are two ‘explain social studies’ 
categories. These two categories were added in order to reflect activities that could not fit 
directly under any of the Process Learning Outcomes, but whose content reflects at least some 
similarity to MSPAP-like activities. The main difference between the ‘explain social studies’ 
categories and the actual Process Learning Outcomes is that the activities receiving a code of 
‘explain social studies’ contain questions that require a short explanation whose answer does not 
require a level of knowledge equal to that defined by the Process Learning Outcomes, or whose 
answer can be extracted directly from reading material. The main difference between the two 
‘explain social studies’ categories is that the ‘explain social studies - reading’ category requires 
that some reading material be referenced in obtaining the answer, and ‘explain social studies - no 
reading’ does not have this reading requirement. Each of the activities was coded for all 
applicable Process Learning Outcomes, including ‘explain social studies’ . 

Process Learning Outcomes by Type of Activity. Table 5 provides the percentages of times 
that each process outcome occurred for instruction, assessment, and MSPAP test preparation 
activities. As indicated in this table, the “Skills and Processes” learning outcomes was coded 
frequently, with 66% of the instruction activities, 58% of the assessment activities, and 88% of 
the test preparation activities being coded for one or more of the indicators of this outcome. 
These percentages are higher than those relating to the other Process Learning Outcomes for each 
type of activity. In addition, within the “Skills and Processes” learning outcomes, larger 
percentages of the tasks required the student to ‘obtain and use textual information’, ‘use print 
and non-print media’, and ‘examine issues, events and problems’. Only small percentages of the 
tasks assessed ‘decision making processes’ or ‘group skills’. These relationships also hold across 
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the type of activity. The percentages of tasks related to “Valuing Self and Others” are much 
higher than the percentages related to “Understandings and Attitudes”. Again, these relationships 
hold across the type of material. 



Table 5. 

Process Learning Outcomes in Social Studies Classroom Activities 



Instruction 

n=1433 


Assessment 

n=1095 


MSPAP Test Preparation 
n=103 


Skills and Processes 


66% 


58% 


88% 


Obtain and Use Textual Information 


29% 


20% 


52% 


Use of Print and Non-print Media 


38% 


32% 


46% 


Examining Issues, Events or Problems 


25% 


27% 


51% 


Decision Making Processes 


3% 


4% 


8% 


Group Skills 


2% 


1% 


2% 


Valuing Self and Others 


18% 


17% 


32% 


Understandings and Attitudes 


5% 


5% 


13% 


Explain Social Studies - no reading 


8% 


19% 


10% 


Explain Social Studies - reading 


12% 


4% 


5% 


None 


19% 


24% 


4% 



In comparing the percentages for the three types of material, similar percentages of 
instruction and assessment activities reflected the Maryland defined Process Learning Outcomes, 
although higher percentages of instruction than assessment activities required students to ‘obtain 
and use textual information’, and ‘use print and non-print materials’. In comparison to test 
preparation activities, the percentages of instruction and assessment tasks reflecting the learning 
outcomes were smaller. The exceptions to these are ‘decision making processes’ and ‘group 
skills’, which show small percentages across the types of material. 

It should also be noted that across grades, the percentages of tasks containing no Process 
Learning Outcomes were 19% for instruction activities, 24% for assessment activities, and only 
4% for test preparation activities. It should be noted that if an activity included only one of the 
‘explain social studies’ categories, it was considered to contain a Process Learning Outcome. 
These percentages again reflect the similarity between the test preparation activities and MSPAP 
test questions, in that a large percentage of the test preparation activities reflect at least one of the 
Process Learning Outcomes, as do the MSPAP tasks. The instruction and assessment activities 
reflect these outcomes to a lesser extent. 

The percentage of tasks that were coded for ‘explain social studies’ was approximately 20% 
for instruction and assessment activities, and 15% for test preparation activities. Again, activities 
were coded as ‘explain social studies’ because the content of the activities was somewhat similar 
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to MSPAP, but not enough so that they contained the Process Learning Outcomes as defined by 
Maryland. The content of such activities was, however, more reflective of MSPAP-like activities 
than tasks consisting of only selected response items. As a result of including the two categories 
relating to explaining social studies as Process Learning Outcomes, the percentage of activities 
reflecting no Process Learning Outcomes may be understated in the table. This is because tasks 
that were coded only for ‘explain social studies’ were counted as reflecting the learning 
outcomes. The percentages of tasks containing only the Maryland-defined Process Learning 
Outcomes, and not the ‘explain social studies’ categories, were approximately 70%, 63%, and 
91% for the instruction, assessment, and test preparation activities, respectively. 

Process Learning Outcomes by Type of Activity and Grade Level . Table 6 presents the 
percentages of times each of the process outcomes was coded for an activity overall and across 
grades. In general, the results were similar for each of the grade levels. The percentages of 
instruction, assessment, and test preparation activities that were coded for one or more indicators 
of the “Skills and Processes” outcome, as indicated by the rows labeled “Skills and Processes,” 
vary little across grades, with the possible exceptions that the percentages for 2 ^^ and 8* grade 
instruction and assessment activities reflecting this outcome are smaller than percentages for the 
other grades. For instruction activities, the results for each of the grade levels are similar for most 
of the “Skills and Processes” indicators. However, the percentages for ‘examining issues, events, 
or problems’ are higher for the upper than lower grades. Differences in percentages for other 
Process Learning Outcomes in the instruction activities can also be seen between higher and 
lower grades. For example, the percentages for “Valuing Self and Others” are slightly higher for 
7**^ and 8**^ grade (24% and 23%, respectively), than for the lower grades (which have percentages 
ranging from 10 to 17%). Also, for the ‘explain social studies - reading’ category, the 
percentages are again higher for 7* and 8**^ grade (both 22%) than they are for the lower grades, 
which have percentages ranging from 5 to 11%. For assessment activities, the percentages of 
tasks reflecting the process outcomes are again similar across grades. One exception may be the 
‘use of print and non-print material’, which decreased slightly in grades 7 and 8 as compared to 
the lower grades. For the test-preparation activities, the percentage of tasks requiring the student 
to ‘obtain and use textual information’ increased as grade increased, whereas the percentage of 
tasks requiring the ‘use of print and non-print media’ decreased as grade increased. In addition, 
“Valuing Self and Others” was represented in half (50%) of the 8* grade activities, as opposed to 
smaller percentages in 3^ (33%) and 5* (14%) grade activities. While differences between 
grades for which Process Learning Outcomes are represented in the instruction, assessment, and 
test preparation activities are present, overall trends are difficult to determine. In addition, there 
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do not appear to be differences in the percentages of tasks reflecting the Process Learning 
Outcomes for the on- (3, 5, and 8) and off- (2, 4, and 7) grades. 



Table 6. 

Process Learning Outcomes in Social Studies Classroom Activities 



All Grades 

Grades 2"** 3^** 4* 5* 7* 8* 



Instruction 

Skills and Processes 
Obtain and Use Textual Information 
Use of Print and Non-print Media 
Examining Issues, Events or Problems 
Decision Making Processes 
Group Skills 
Valuing Self and Others 
Understandings and Attitudes 
Explain Social Studies - no reading 
Explain Social Studies - reading 
None 

Assessment 

Skills and Processes 
Obtain and Use Textual Information 
Use of Print and Non-print Media 
Examining Issues, Events or Problems 
Decision Making Processes 
Group Skills 
Valuing Self and Others 
Understandings and Attitudes 
Explain Social Studies - no reading 
Explain Social Studies - reading 
None 

MSPAP Test Preparation 
Skills and Processes 
Obtain and Use Textual Information 
Use of Print and Non-print Media 
Examining Issues, Events or Problems 
Decision Making Processes 
Group Skills 
Valuing Self and Others 
Understandings and Attitudes 
Explain Social Studies - no reading 
Explain Social Studies - reading 
None 



66% 


57% 


76% 


71% 


70% 


64% 


59% 


29% 


21% 


23% 


32% 


28% 


33% 


36% 


38% 


39% 


52% 


42% 


39% 


33% 


27% 


25% 


11% 


16% 


25% 


34% 


35% 


29% 


3% 


3% 


4% 


4% 


5% 


2% 


3% 


2% 


1% 


0% 


1% 


2% 


4% 


2% 


18% 


16% 


10% 


17% 


15% 


24% 


23% 


5% 


1% 


4% 


2% 


10% 


4% 


7% 


8% 


5% 


9% 


6% 


9% 


13% 


9% 


12% 


5% 


6% 


11% 


6% 


22% 


22% 


19% 


30% 


15% 


14% 


17% 


18% 


21% 


58% 


48% 


70% 


58% 


67% 


51% 


54% 


20% 


10% 


18% 


14% 


25% 


22% 


27% 


32% 


36% 


42% 


38% 


32% 


23% 


24% 


27% 


15% 


29% 


22% 


33% 


29% 


32% 


4% 


2% 


5% 


2% 


3% 


3% 


7% 


1% 


2% 


1% 


1% 


1% 


3% 


1% 


17% 


13% 


12% 


14% 


17% 


28% 


16% 


5% 


1% 


3% 


5% 


7% 


4% 


11% 


19% 


13% 


18% 


27% 


19% 


17% 


21% 


4% 


1% 


2% 


3% 


4% 


8% 


7% 


24% 


34% 


17% 


17% 


17% 


32% 


28% 


88% 





87% 





90% 





89% 


52% 


— 


41% 


— 


52% 


— 


68% 


46% 


— 


54% 




41% 


— 


36% 


51% 


— 


50% 


— 


41% 


— 


61% 


8% 


— 


4% 


— 


10% 


— 


11% 


2% 


— 


2% 


-- 


0% 


— 


4% 


32% 


— 


33% 


— 


14% 


— 


50% 


13% 


— 


13% 


— 


14% 


— 


11% 


10% 


— 


7% 


— 


14% 


— 


11% 


5% 


— 


2% 


— 


3% 


— 


11% 


4% 


— 


7% 


— 


0% 


— 


4% 



Extent to which classroom activities reflect the Process Learning Outcomes. Table 7 
presents for the instruction, assessment and test preparation activities the percentages of the 



activities for which all, part, or none of the task contained Process Learning Outcomes. Those 
activities for which none of the task reflected the Process Learning Outcomes consisted of only 
selected response or short answer items, and were coded as ‘distinct non-MSPAP items’. 
Activities for which part of the task contained Process Learning Outcomes were most often 
classified as both ‘one task with several items related to the same context’ and ‘distinct non- 
MSPAP items’. To be placed into this category, part of an activity had to consist of selected 
response or short answer items. The ‘explain social studies’ category was included for those 
tasks that required some explanation of social studies, but not enough to reflect the learning 
outcomes as defined by MSPAP. These tasks were more like MSPAP than tasks containing only 
selected response items, and in general consisted of a series of short explanations. Finally, 
activities that reflected any of the Process Learning Outcomes and did not consist of any selected 
response items fell into the last category, ‘all of task has Process Learning Outcomes’. 



Table 7. 

Extent to Which Activities Reflect Process Learning Outcomes 





All 

grades 


2nd 


3rd 


Grade 

4th 5 * 


'yth 


8 * 


Instruction 


Task has no Process Outcomes 


19% 


30% 


15% 


14% 


17% 


18% 


21 % 


Explain Social Studies 


11 % 


7% 


8 % 


10 % 


9% 


16% 


16% 


Part of Task has Process Outcomes 


17% 


8 % 


17% 


10 % 


14% 


29% 


22 % 


All of Task has Process Outcomes 


53% 


56% 


61% 


66 % 


60% 


38% 


42% 


Assessment 


Task has no Process Outcomes 


24% 


34% 


17% 


17% 


17% 


32% 


28% 


Explain Social Studies 


13% 


12 % 


11 % 


20 % 


11 % 


11 % 


15% 


Part of Task has Process Outcomes 


18% 


8 % 


21 % 


21 % 


18% 


16% 


23% 


All of Task has Process Outcomes 


45% 


45% 


51% 


41% 


54% 


41% 


34% 


MSPAP Test Preparation 


Task has no Process Outcomes 


4% 


— 


7% 


— 


0 % 


— 


4% 


Explain Social Studies 


6 % 


— 


4% 


— 


7% 


— 


7% 


Part of Task has Process Outcomes 


14% 


— 


11 % 


— 


10 % 


— 


21 % 


All of Task has Process Outcomes 


77% 


“ 


78% 


— 


83% 


“ 


68 % 



As can be seen from the table, over half of the instruction activities (53%), almost half of 
the assessment activities (45%), and over three-fourths of the test preparation activities (77%) 
reflected the Process Learning Outcomes throughout the task. The percentages of instruction and 
assessment activities that were classified as ‘task has no Process Outcomes’ and ‘explain social 
studies’ were similar (19% and 11% for instruction, and 24% and 13% for assessment activities, 
respectively). The percentages test preparation activities that fell into these two categories, 
however, were much smaller (4% and 6 %, respectively). Across types of activities, the 



percentages of classroom activities for which part of the task reflected the Process Learning 
Outcomes were similar. In general, the percentages for instruction and assessment activities for 
which all, part, or none of the activity reflected the Process Learning Outcomes were similar to 
each other. As compared to test preparation activities, smaller percentages of instruction and 
assessment activities reflected the Process Learning Outcomes throughout the activities, and 
larger percentages of instruction and assessment activities than test preparation activities did not 
reflect any of the Process Learning Outcomes. 

In comparing the results for each grade level, the percentage of instruction activities that 
contained no Process Learning Outcomes was higher in grade 2 than in the other grades. For 
assessment activities, the percentages of tasks that contained no Process Learning Outcomes were 
higher in grades 2, 7, and 8 than in the other grades. This would indicate that in assessing 
students, teachers, to some extent, use selected response/short answer formats. For instruction 
activities, the percentages of activities labeled ‘all of task has process outcomes’ were lower for 
grades 7 and 8 than for the other grades. For assessment activities, the percentages of tasks 
falling in this category was much more consistent across the grades. For test preparation 
activities, the percentages across grades were relatively consistent and small for the ‘task has no 
Process Outcomes’ and ‘explain social studies’ categories. A higher percentage of grade 8 
activities had part of the task reflecting the process learning outcomes than the lower grades. The 
percentages of instruction, assessment, and test preparation activities for which the entire task 
reflected the process outcomes decreased somewhat for the 7* and 8* grades (8* grade for test 
preparation activities) as compared to the lower grades. 

Comparisons of Process Learning Outcomes by grade level and type of activity . In order to 
determine if significant differences in the percentages of instruction and assessment activities 
reflecting the Process Learning Outcomes were present, a repeated measures analysis of variance 
with one between factor (grade) and one within factor (type of activity) was conducted. Table 8 
shows the average proportion per teacher of social studies instruction and assessment activities 
that reflected at least one Process Learning Outcome. The proportions are presented for each 
grade level. For this analysis, only those teachers who sent in both instruction and assessment 
activities were included, which represents 57% of all social studies teachers who sent in 
classroom activities (172 out of 304) '. After transforming the data using an arcsine 
transformation, the results of the analysis for the Process Learning Outcomes indicate a 

' MSPAP test preparation activities were not included in this analysis because of the small sample size. A 
comparison among the three types of activities (instruction, assessment, and MSPAP test preparation) 
would have reduced the sample to only 52 teachers, which represents only 17% of all social studies 
teachers. 
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significant difference between grades (F(5, 166)=3.526, p=.005). No significant difference 
within teachers for instruction versus assessment activities (F(l, 166)=1.017, p=.315) was 
obtained. The interaction between grade level and type of activity was also not significant (F(5, 
166)=1.062, p=.384). This indicates that the average proportion of activities that reflected at least 
one Process Learning Outcome was not significantly different for instruction and assessment 
activities, but the differences across grades were significant. Tukey HSD post-hoc analyses were 
conducted in order to determine which differences between grades were significant. The only 
significant differences between grades occurred between grades 2 and 4, and grades 2 and 5. A 
larger proportion of 4* (.84) and 5* (.81) grade activities reflected at least one Process Learning 
Outcome as compared to 2"‘* (.63) grade (p=.005 and .027, respectively). No other differences 
between grades were significant. It should be noted that for this analysis, those tasks that 
contained only ‘explain social studies’ were considered to contain at least one Process Learning 
Outcome, even though ‘explain social studies’ was not a Process Learning Outcome as defined by 
Maryland. However, a similar analysis was carried out in which activities containing only 
‘explain social studies’ and no other process outcomes were not counted as containing at least one 
Process Learning Outcome. The results of this analysis were consistent with those just presented. 



Table 8. 

Process Learning Outcomes for Social Studies Activities by Grade and Type of Activity 





All 

grades 

(n=172) 


2nd 

grade 

(n=27) 


— p— 
grade 
(n=34) 


grade 

(n=23) 


grade 

(n=26) 


'ylh 

grade 

(n=28) 


8* 

grade 

(n=34) 


At least one Process Learning Outcome 
Instruction activities 


.77 


.61 


.80 


.84 


.78 


.77 


.82 


Assessment activities 


.74 


.65 


.77 


.84 


.85 


.64 


.72 



Maryland Content Learning Outcomes 

The classroom instruction, assessment, and MSPAP test preparation tasks were also coded in 
terms of their content emphasis. The Maryland Learning Outcomes provided the basis for the 
classification and are provided in Table 9. They are political systems, geography, ‘peoples of the 
nation/world’, and economics. It should be noted that the ‘general social studies’ category was 
not one of Maryland’s categories. For the purpose of this analysis, ‘general social studies’ was 
coded for a task when a more specific content area could not be identified, but the task related to 
social studies. For example, if a task was related to WWn, but it could not be determined if the 
task was related to the geography, people, or economics of the war, the ‘general social studies’ 
category was coded. If this category was chosen for a task, no other content area was selected. 
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More than one content area could have been selected for a task if the information asked for was 
fairly detailed and related specifically to one of the content areas. Otherwise, the main content 
area being assessed in the task was selected. All other categories besides ‘general social studies* 
were defined according to the Maryland Learning Outcomes. 

Content Learning Outcomes Across Type of Activity. The percentages of times each of the 
content learning outcomes was coded for an activity are presented in Table 9. The percentages 
reflect the percentage of times each of the content outcomes was coded for an activity regardless 
of how many content outcomes were coded per activity. The percentages are provided across the 
type of material, because the differences in instruction, assessment, and test preparation activities 
that represented each of the content areas were slight. Overall, the percentages for geography and 
‘peoples of the nation/world’ are the highest. More than half of activities contained content 
relating to geography. Very few of the tasks were classified as being ‘general social studies’, 
indicating that most often, the content reflected in the task could be classified as one or more of 
the Content Learning Outcomes defined by Maryland. 



Table 9. 

Content Learning Outcomes for Social Studies Classroom Activities 





All 

grades 


2nd 


3rd 


Grades 

4th 5th 


'yth 


8* 


Political Systems 


24% 


10% 


7% 


15% 


27% 


32% 


48% 


Geography 


51% 


60% 


65% 


55% 


46% 


47% 


35% 


Peoples of the NationAVorld 


38% 


24% 


26% 


40% 


39% 


59% 


42% 


Economics 


19% 


21% 


25% 


17% 


15% 


19% 


17% 


General Social Studies 


2% 


1% 


2% 


2% 


2% 


2% 


1% 



Content Learning Outcomes by Grade. Table 9 also shows the percentages of activities 
coded for each content outcome for each grade level. In general, the percentages of tasks 
reflecting ‘political systems’ are higher in the upper than lower grades. Higher grades also had 
higher percentages of tasks reflecting ‘peoples of the nation/world’ than the lower grades. In 
contrast, higher percentages of activities in the lower than upper grades had geography content. 
The percentages of activities that contained economics content were similar across grades. 



Other Task Features 

In addition to the process and content outcomes, other features of the classroom activities 
were coded including work groups, integration with other subject areas, use of resources, and the 
type of response required by students. The analysis of instruction, assessment, and test 
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preparation activities by these features provide additional information for describing their 
similarity to MSPAP tasks. 

Group Work. The Maryland Learning Outcomes indicate that providing students with 
opportunities to work collaboratively with others is an important aspect of the learning 
environment. Therefore, the classroom activities were coded as to whether they involved 
individual, pair, or group work. Table 10 presents the percentages of instruction, assessment, and 
test preparation activities that required individual, pair, or group work. Only one category per 
activity was chosen for the type of work group required. As can be seen, the majority of 
instruction, assessment, and test-preparation activities required only individual work. Also, only 
small percentages of the instruction, assessment, and test preparation activities required only 
group work. Overall, a higher percentage of instruction (15%) than assessment (9%) activities 
called for both individual and group work. The percentages of test-preparation activities 
requiring individual and group work were relatively high (30%). The percentages of activities 
sent by teachers that required only class discussion were small. It should be noted that it is 
possible that the percentages of pair or group work for the social studies classroom activities is 
actually higher than is presented since group work was coded only if it was explicitly indicated on 
the activity that students were to work together. 



Table 10. 

Type of Work Group for Social Studies Classroom Activities 





Instruction 

n=1433 


Assessment 

n=1095 


MSPAP Test Preparation 
n=103 


Individual 


79% 


87% 


62% 


Group 


6% 


4% 


8% 


Individual and Group 


15% 


9% 


30% 


Class Discussion 


1% 


0% 


0% 



Integration with Other Subject Areas . The majority of the MSPAP tasks that assess social 
studies are integrated with other subject areas including, reading, writing, language usage, science 
and mathematics. Therefore, the social studies classroom activities were analyzed in terms of 
whether they were integrated with other subject areas. The integration categories include those 
just mentioned. Two categories for reading integration were coded, ‘reading textbook or 
nonauthentic material’, and ‘reading authentic material’. Authentic material would be, for 
example, newspaper or magazine articles, books other than the textbook, or other research 
materials. Nonauthentic material would be readings from the text, or paragraphs that appear on a 
worksheet with no reference. Integration with writing was selected if the activity required 
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students to provide long explanations, or to write an article, letter, speech, research report or 
journal entry. Language usage was selected if it was indicated in the task that the student should 
use correct grammar, spelling or punctuation. Science and mathematics were selected if the 
content of the task included content related to these subject areas in addition to social studies 
content. 

Table 1 1 indicates the percentage of times each category of integration was coded for each 
type of activity when one or more integration categories were coded per activity. The table also 
indicates the percentage of activities that required no integration. Across grades, 62% of the 
social studies instruction activities and 52% of the assessment activities were integrated with 
other subject areas, while 89% of the test preparation activities were integrated with other subject 
areas. The most common forms of integration regardless of the type of activity were within the 
writing area. Overall, 31% of the instruction activities, 38% of the assessment activities, and 
70% of the MSPAP test preparation activities involved writing. As an example, for the instruction 
activities, 22% were coded for long explanations, 2% were coded for being a journal or log, and 
11% were coded for being an article, letter, or research report. It should be noted again that an 
activity could be coded for more than one type of writing. Higher percentages of test preparation 
activities than instruction and assessment activities also involved reading integration. Assessment 
activities had the lowest percentage of reading integration. Instruction and assessment activities 
had similar percentages of mathematics integration, (8% and 6%, respectively), while a higher 
percentage of test preparation activities were integrated with this content area (19%). 
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Table 11. 

Integration of other Subject Areas in Social Studies Classroom Activities 



Instruction 

n=1433 


Assessment 

n=1095 


MSPAP Test 
Preparation 
n=103 


Reading Textbook or Nonauthentic Material 


25% 


12% 


33% 


Reading Authentic Material 


21% 


15% 


31% 


Writing 


31% 


38% 


70% 


Long Explanations 


22% 


25% 


45% 


Journal 


2% 


1% 


4% 


Article/Letter/Research Report 


11% 


14% 


26% 


Language Usage 


8% 


11% 


31% 


Science 


5% 


5% 


7% 


Mathematics 


8% 


6% 


19% 


No Integration 


38% 


48% 


11% 



Use of Resources. Many of the MSPAP tasks require students to utilize resources available 
inside and outside of the classroom. Table 12 indicates the percentages of classroom instruction, 
assessment, and test preparation activities that required the use of such resources. It should be 
noted that each activity could be coded for more than one resource. At least one resource was 
coded for 53% of the instruction activities, 39% of the assessment activities, and 60% of the test 
preparation activities. For instruction, assessment, and test preparation activities, only one 
category for resources was selected in 44%, 31%, and 46% of the tasks, respectively. The 
resources that were required the most across the classroom activities were ‘maps, globes and 
atlases’, as would be expected for the social studies subject area. Relatively high percentages of 
activities required ‘references’, ranging from 10 - 14% for the type of activities. A smaller 
percentage of assessment than instruction activities required the use of ‘books, poems, plays or 
stories’, while a higher percentage of test preparation activities utilized these as a resource. Also, 
fewer instruction than test preparation materials used these resources. 
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Table 12. 

Resources Used with Social Studies Classroom Activities 





Instruction 

n=1433 


Assessment 

n=1095 


MSPAP Test Preparation 
n=103 


Maps/Globes/ Atlases 


29% 


22% 


33% 


References (encyclopedias, articles, etc.) 


13% 


10% 


14% 


Books/Poems/Plays/Stories 


11% 


5% 


17% 


Artifacts/Memorabilia 


7% 


4% 


11% 


Viewing Films or Videotapes 


4% 


1% 


1% 


Field Trip/Archeology Dig/Interviews 


2% 


2% 


2% 


Artwork/Art Supplies 


2% 


2% 


2% 


Computer/Intemet/Calculator 


2% 


2% 


5% 


None 


47% 


61% 


40% 



Response Required of Student . The MSPAP tasks that are scored for social studies require 
students to respond in a variety of ways including providing short answers, short and long 
explanations, and visual representations. For the analysis of the classroom materials, a response 
that is a word, phrase, or definition was coded as a short answer. A short explanation was defined 
as one to three written sentences, or posing questions, while a long explanation would be such 
responses as a paragraph or essay. Visual representations include such responses as sketches, 
drawings and posters, and graphic organizers were such responses as Venn diagrams, character 
maps, or outlines that serve as language arts based organizers. 

Table 13 indicates the percentage of times an activity was coded for each of the response 
types regardless of the number of response types that were coded per activity. A higher 
percentage of assessment activities (36%) than both instruction (15%) and test preparation (11%) 
activities contained multiple choice/matching format types, while similar percentages of each of 
the types of activities contained short answer formats. Similar percentages of instruction and 
assessment activities required short and long explanations, while the percentages of test 
preparation activities requiring these two answer types were much higher. In general, the 
percentage of instruction and assessment activities containing short explanations, long 
explanations, articles, and speeches are similar to each other, and smaller than the corresponding 
percentages for test preparation activities. As might be expected, the response types that were 
coded for the MSPAP test preparation activities, as compared to those coded for the instruction 
and assessment activities, are more similar to the response types of MSPAP tasks. 
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Table 13. 

Response Type for Social Studies Classroom Activities 





Instruction 

n=1433 


Assessment 

n=1095 


MSPAP Test 
Preparation 
n=103 


Multiple Choice/Matching 


15% 


36% 


11% 


Short Answer 


43% 


40% 


46% 


Short Explanation 


36% 


37% 


62% 


Long Explanation 


22% 


25% 


45% 


Article/Letter/Research Report 


11% 


14% 


26% 


Speech/Debate/Oral Presentation 


7% 


6% 


18% 


Joumal/Log 


2% 


1% 


4% 


Class Discussion 


16% 


4% 


16% 


Visual Representation/Diorama 


20% 


19% 


22% 


Graphic OrganizerA^enn Diagram/Timeline/Outline 


20% 


13% 


30% 


Filling in or Creating Maps 


19% 


14% 


19% 


Charts/Tables/Graphs/Glyphs 


19% 


12% 


33% 


Other 


3% 


2% 


2% 



♦Other; archeological digs, games 



Similarity to MSPAP Tasks 

Each of the classroom instruction, assessment, and MSPAP test preparation activities was 
coded with respect to its overall simileirity to MSPAP tasks. In particular, the process needed for 
solution, the type of responses required of students, and the format of the responses were 
considered in order to classify the activities according to one or more MSPAP-like levels. The 
lowest level includes those activities that were considered to be ‘not at all like MSPAP’ . The 
other four levels include activities that are similar to MSPAP tasks to some extent: MSPAP-like 
1, MSPAP-like 2, MSPAP-like 3, and MSPAP-like 4. Figure 1 provides descriptors of a typical 
task for each of the five levels on the MSPAP-like scale. 

It should be noted that in the original coding of the data, more than one MSPAP level could 
have been selected for a single activity. For instance, if the activity consisted of a set of multiple 
choice items and one significant long explanation, the activity would have received a code of ‘not 
at all like MSPAP’ and an additional MSPAP-like code. However, at the time of the coding, the 
MSPAP level that was emphasized in the activity was also coded. So, for the above example, the 
activity would have received an emphasis on the ‘not at all like MSPAP’ aspects, even though in 
the long explanation, some of the Process Learning Outcomes could have been present. The 
results presented in this report assume that each activity received a single MSPAP-like level, that 
is, the results presented are based on the MSPAP level that the activities most emphasized. Also, 
if an activity reflected two MSPAP-like levels equally, the higher of the two levels was chosen to 
represent the activity as a whole. 
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Figure 1. 

Description of the Five Levels of the Social Studies MSPAP Scale 



MSPAP scale Levels 


Descriptors of Levels 


Not at all like MSPAP 


Multiple-choice, matching, true false, only short answer formats 
Classified as 'distinct non-MSPAP items’ 

Do not reflect any of the Process Learning Outcomes including explain social studies 




Example: 

Answering a set of selected response items 


MSPAP-like 1 


Do not reflect any of the Maryland defined Process Learning Outcomes, but always 
require some explanation of social studies 

Require short explanations that may or may not be based on reading 




Example: 

Reading a paragraph of text, and extracting information directly from the text in order 
to answer one or more questions 


MSPAP-like 2 


Reflect at least one, and usually not more than two Process Learning Outcomes 
Classified as ‘one task with several items related to the same context* or ‘only one 
item’ 

Activities need not be set in a realistic context 




Examples: 

Creating a map 

Writing a single long explanation to a question 
Answering a two questions with short explanations 


MSPAP-like 3 


Reflect a number of the Process Learning Outcomes 

Classified as ‘one task with several items related to the same context’ or ‘only one 
item’ 

Typically contain at least three short explanations or one long explanation along with 
a reading 

Similar to MSPAP tasks in terms of process and format, but not as lengthy 
Activities are related to the same context or social studies theme 




Example: 

Reading a passage about a Supreme Court case and responding with three short 
explanations 

The questions are related to the same context, but the activity is not as lengthy as a 
MSPAP task, so it would be classified as MSPAP-like 3 


MSPAP-like 4 


Reflect a number of the Process Learning Outcomes 

Classified as ‘one task with several items related to the same context’ 

Typically contain at least four short explanations or two long explanations 
Typically require students to respond to five or more items related to the same 
context 




Example: 

Answering several questions related to different aspects of slavery after reading a 
passage, with each question requiring a long explanation, and often an entire 
paragraph 

Since the questions are related to the same context and discuss different aspects of 
slavery, and the responses are extensive, the activity would be classified as MSPAP- 
like 4 
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Similarity to MSPAP tasks bv Type of Activity. As is indicated in Table 14, similar 
percentages of instruction and assessment tasks were found at the MSPAP-like 1, MSPAP-like 3, 
and MSPAP-like 4 levels. For both instruction and assessment activities, 29% of the activities 
were coded at the MSPAP-like 3 or MSPAP-like 4 levels, with the majority of these receiving a 
code of MSPAP-like 3. Differences in the percentages of instruction and assessment activities 
that were coded at the ‘not at all like MSPAP’ and MSPAP-like 2 levels were found. A higher 
percentage of assessment (34%) than instruction (22%) were coded at the ‘not at all like MSPAP 
level’, while a lower percentage of assessment (27%) than instruction (37%) activities were coded 
at the MSPAP-like 2 level. 

The percentages of MSPAP test preparation activities coded for each level of the MSPAP 
scale differed in comparison to the instruction and assessment activities. As might be expected, 
the 3'**, 5*, and 8* grade MSPAP test preparation activities, as compared to the instruction and 
assessment activities, are more similar to MSPAP tasks as indicated by the ‘all grades’ column. 
Smaller percentages of test preparation activities than both instruction and assessment activities 
were coded at the ‘not at all like MSPAP’, MSPAP-like 1, and MSPAP-like 2 levels. Larger 
percentages of test preparation activities than the other two types were coded at the MSPAP-like 
3 and MSPAP-like 4 levels, with a large difference being found at the MSPAP-like 4 level 
between the test preparation (39%), and instruction (9%) and assessment (8%) activities. In 
general, it is apparent that the MSPAP test preparation activities reflect the MSPAP tasks more 
than the instruction and assessment activities. It also appears that assessment activities reflect the 
MSPAP tasks to a lesser extent than instruction activities. 
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Table 14. 

MSPAP-like Levels for Social Studies Classroom Activities 





All 

grades 


2nd 


3rd 


Grade 

4U1 5U1 


'yth 


8* 


Instruction 


Not at all like MSPAP 


22% 


34% 


19% 


14% 


20% 


21% 


25% 


MSPAP-like Levels 


MSPAP-like 1 


12% 


5% 


8% 


13% 


9% 


19% 


18% 


MSPAP-like 2 


37% 


39% 


52% 


41% 


36% 


28% 


26% 


MSPAP-like 3 


20% 


16% 


15% 


23% 


21% 


22% 


22% 


MSPAP-like 4 


9% 


7% 


7% 


8% 


14% 


11% 


9% 


Assessment 


Not at all like MSPAP 


34% 


40% 


25% 


30% 


24% 


41% 


43% 


MSPAP-like Levels 


MSPAP-like 1 


10% 


8% 


9% 


15% 


11% 


8% 


10% 


MSPAP-like 2 


27% 


33% 


35% 


29% 


29% 


24% 


15% 


MSPAP-like 3 


21% 


13% 


21% 


18% 


28% 


18% 


25% 


MSPAP-like 4 


8% 


6% 


10% 


8% 


8% 


9% 


7% 


MSPAP Test Preparation 


Not at all like MSPAP 


9% 


— 


13% 


— 


3% 


— 


7% 


MSPAP-like Levels 


MSPAP-like 1 


6% 


— 


4% 


— 


3% 


— 


11% 


MSPAP-like 2 


21% 


— 


20% 


— 


28% 


— 


18% 


MSPAP-like 3 


25% 


— 


17% 


— 


31% 


— 


32% 


MSPAP-like 4 


39% 


” 


46% 


- 


35% 


- 


32% 



Similarity to MSPAP tasks by Type of Activity and Grade. Table 14 also provides the results 
for each grade level. For the instruction activities, differences across grades in the percentages of 
tasks at the ‘not at all like MSPAP’ level were rather small, with the exception of grade 2, whose 
percentage was larger than the other grades. Also, higher percentages of 7* and 8* grade tasks 
received a code of MSPAP-like 1 as compared to grades 2-5. In addition, smaller percentages of 
2"“* (23%) and 3^“* (22%) grade instruction tasks were classified at either the MSPAP-like 3 or 4 
levels than percentages for the other grades (which range from 31% - 35%). The percentages of 
tasks at all grade levels receiving a code of MSPAP-like 4 were smaller than the percentages 
receiving a code of MSPAP-like 3. 

For assessment activities, the percentages of 2"“*, 7*, and 8* grade tasks falling into the ‘not at 
all like MSPAP’ level were higher that the percentages in the other grades. Differences across 
grades for the MSPAP-like 1 level were small. For tasks receiving codes of MSPAP-like 3 or 4, 
differences can be seen between the on- and off-grades. The percentages of tasks at these 
MSPAP-like levels were larger in the on-grades (31%, 36%, and 32% for grades 3, 5, and 8, 
respectively) than off-grades (19%, 26%, and 27%, for grades 2, 4, and 7, respectively). As with 
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instruction activities, the percentages of assessment tasks at all grade levels receiving a code of 
MSPAP-like 4 were smaller than the percentages receiving a code of MSPAP-like 3. 

With regard to MSPAP test preparation activities, a higher percentage of than 5* or 8* 
grade activities were coded as ‘not at all like MSPAP’. Similar percentages of (63%), 5* 
(66%), and 8* (64%) grade test preparation activities were coded at either the MSPAP-like 3 or 4 
levels. In addition, a higher percentage of 3^‘* than 5* and 8* grade activities obtained a code of 
MSPAP-like 4. 

In general, the differences in the percentages of activities at each MSPAP level at the grade 
levels were different for instruction, assessment, and test preparation activities. The only 
differences between on- and off-grades seemed to be in assessment activities that were coded at 
the MSPAP-like 3 and 4 levels. Another interesting finding was the decrease in the percentage of 
test preparation activities at the MSPAP-like 4 level across grades. 

Comparisons bv grade level and type of activity . In order to determine if there were 
differences in the level of similarity to MSPAP tasks between grade levels and within instruction 
and assessment activities, a repeated measures analysis of variance with one between factor 
(grade) and one within factor (type of activity) was conducted. Table 15 shows an average 
MSPAP level for teachers at each grade level. For this analysis, only those teachers who sent in 
both instruction and assessment activities were included, which represents 57% of all social 
studies teachers who sent in classroom activities (172 out of 304). Similar to the previous 
repeated measures analysis for Process Learning Outcomes, the MSPAP test preparation activities 
were not included in this analysis because of the small sample size. The results of the analysis for 
the MSPAP scale indicate no significant difference within teachers for instruction versus 
assessment activities (F(l, 166)=1.155, p=.284). In addition, no significant difference occurred 
between grade levels (F(5, 166)=1.916, p=.094). The interaction between grade and type of 
activity was also not significant (F(5, 166) = 1.263, p=.282). This indicates that the average 
MSPAP level per teacher for the instruction and assessment activities was not significantly 
different, nor were the differences in average MSPAP level assigned significant across grades. 

Table 15. 

Similarity to Actual MSPAP Social Studies Tasks bv Grade and Type of Activity 





All 


2"“ 


3rd 




5“ 









grades 


grade 


grade 


grade 


grade 


grade 


grade 




(n=172) 


(n=27) 


(n=34) 


(n=23) 


(n=26) 


(n=28) 


(n=34) 


Instruction activities 


2.61 


2.23 


2.60 


2.84 


2.82 


2.58 


2.65 


Assessment activities 


2.54 


2.40 


2.62 


2.51 


2.88 


2.28 


2.54 
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Discussion 



This paper presented the results of analyses investigating the similarity of classroom 
materials collected from social studies teachers in the state of Maryland during the 1997-98 
school year to the Maryland Learning Outcomes and MSPAP. The similarity of the classroom 
materials to the Maryland Learning Outcomes was considered separately for Process Learning 
Outcomes and Content Learning Outcomes. With respect to the Process Learning Outcomes, it 
was found that indicators of the “Skills and Processes” outcome, ‘obtain and use textual 
information’, ‘use of print and non-print media’, and examining issues, events or problems’ were 
the process outcomes present most often across instruction, assessment, and test preparation 
activities. “Valuing self and others” was the next most often found process outcome, followed by 
“Understandings and Attitudes”. It was also found that the percentage of instruction and 
assessment tasks reflecting the process outcomes were similar to each other, but were much 
smaller than the percentage of test preparation activities reflecting the Process Learning 
Outcomes. However, approximately 75% of the instruction and assessment activities did reflect 
at least one Process Learning Outcome (including ‘explain social studies’). While some 
differences in the percentages of activities reflecting the Process Learning Outcomes across 
grades were present, it was difficult to determine trends in these differences. It was apparent, 
however, in examining the extent to which the classroom activities reflected the Process Learning 
Outcomes, that the percentages of grade 2 instruction and assessment activities that were 
classified as containing no Process Learning Outcomes were higher than the percentages in the 
other grades. Further, the percentages of grade 7 and 8 assessment activities that reflected no 
process outcomes were somewhat larger than the other grades. It did not appear that there were 
differences between the on- and off-grades. 

Further analysis of the Process Learning Outcomes showed that there were no significant 
differences in the percentages of instruction and assessment activities reflecting at least one of the 
Process Learning Outcomes. Test preparation activities were not included in this analysis, 
because including activities of this type would have greatly reduced the sample size. These 
results are not surprising based on the fact that similar percentages of instruction and assessment 
activities reflected each of the Process Learning Outcomes, as was stated previously. This further 
analysis of the Process Learning Outcomes did show that there was a significant difference across 
grades in the percentages of instruction and assessment activities that reflected at least one of the 
outcomes. It was found that grade 2 had smaller percentages of activities reflecting at least one 
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Process Learning Outcome than grades 4 and 5. These results are consistent with the percentages 
discussed above. 

With respect to the Content Learning Outcomes, there were some differences in the 
percentages of activities reflecting the different content areas across grades, but little difference in 
the percentages of instruction, assessment, and test preparation activities that reflected the content 
areas within grades. 

Activities that required both individual and group work were not commonly seen in the 
instruction or assessment activities, but were more common in the test preparation activities. This 
could be due in part to the fact that teachers did not specify on the instruction and assessment 
activities that group work was part of the activity. If this specification was not made in writing, 
the task was treated as requiring individual work only. 

More of the MSPAP test preparation activities required integration with other subject 
areas than both the instruction and assessment activities, with more of the instruction than 
assessment activities requiring integration of some sort. Writing integration was the most 
common form of integration across the type of activity, as might be expected in the social studies 
subject area, but the percentage of test preparation activities that were integrated with writing was 
much higher than the percentage of instruction or assessment activities that required writing 
integration. Assessment activities also did not utilize reading integration nearly as much as 
instruction or test preparation activities. 

With respect to the response types that the activities required, more assessment than 
either instruction or MSPAP test preparation activities utilized multiple choice/matching formats 
in at least some part of the activity. Similar percentages of instruction and assessment activities, 
smaller than the percentages for test preparation activities, required short and long explanations, 
articles, speeches, or language arts based graphic organizers. 

Many of these findings point to the conclusion that while the test preparation activities 
used by the Maryland teachers are quite similar to the MSPAP tasks themselves, and therefore 
represent the Maryland Learning Outcomes quite well, this similarity is carried over to a much 
smaller extent in the instruction and especially, at least with respect to some features, the 
assessment activities. 

In further considering the relationship between the classroom activities and the Maryland 
Outcomes, an important comparison can be made between the overall similarity of the activity to 
the tasks on the MSPAP. This comparison was made in a number of ways throughout this paper. 
Initially, the format of the classroom activities was considered. It was found that an 
overwhelming majority of the MSPAP test preparation activities (91%) consisted of a format 
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similar to the MSPAP tasks, which was classified in the paper as being ‘one task with several 
items related to the same context’. A larger percentage of instruction than assessment activities 
had this format. In addition, only a small percentage of test preparation activities consisted of 
‘distinct non-MSPAP items’, while almost half of the assessment activities contained, at least in 
part, items of this format. 

Another, and the most encompassing measure that was used to relate the classroom 
activities to the Maryland Learning Outcomes and the MSPAP tasks, was the MSPAP-like scale. 
Overall, it was found that, as expected, few of the MSPAP test preparation activities fell into the 
two lowest categories on this scale. Further, approximately 40% of the test preparation activities 
fell into the highest MSPAP-like category. Instruction and assessment activities showed different 
patterns. Almost one-fourth of the instruction activities and one-third of the assessment 
activities fell at the lowest MSPAP-like level, and less than 10% of these types of activities fell at 
the highest MSPAP-like level. Approximately 20% of the instruction and assessment activities 
did, however, fall into the MSPAP-like 3, or second highest, level. Some differences in the 
percentages of activities falling into the MSPAP-like levels across grades were found. Interesting 
differences for instruction activities include higher percentages of MSPAP-like 1 activities in 7th 
and 8th grades, and smaller percentages of MSPAP-like 2 activities in these grades. For 
assessment activities, higher percentages of grades 2, 7, and 8 activities fell into the lowest 
MSPAP category. This result is consistent with the percentages of assessment tasks in these 
grades containing no Process Learning Outcomes. Another interesting finding was that the 
percentages of on-grade assessment activities in either the MSPAP-like 3 or 4 level were higher 
than the off-grade percentages. This is one of the few instances that differences between on- and 
off-grade levels were found. 

It was also found, in comparing the teachers’ average MSPAP level for instruction and 
assessment activities at each grade level, that there was no significant difference in the average 
MSPAP score for instruction and assessment activities, and also that there were no significant 
differences in the average MSPAP scores across grades. It should be noted again that test 
preparation activities were not included in this analysis because the sample size would have been 
greatly reduced if they were included. 

The analysis of the social studies classroom materials sent in by teachers in the state of 
Maryland during the 1997-98 school year suggests that the instruction and assessment classroom 
activities are similar to each other in their representation of the Maryland Learning Outcomes and 
similarity to MSPAP-like tasks. These activities are less similar to the learning outcomes and 
tasks, however, than the MSPAP test preparation activities. 
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